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1  TECHNICAL  PROGRESS/SIGNIFICANT  ACCOMPLISHMENTS 
AND  EVENTS  OVER  THE  CURRENT  REPORTING  PERIOD 
(PLUS  HISTORIC  PROGRESS) 

February  2011 

•  Internal  kick-off  completed  February  3-4  in  Los  Angeles,  with  all  team  members 

•  Presented  work  on  this  project  in  talks  at  HSCB  Focus  201 1  conference,  February  7-10 

o  Lewis  Johnson:  Model-Based  Cultural  Competence  Training 
o  Alicia  Sagae:  Usability  in  MMT  for  Up-To-Date  Cultural  Competence  Training 
o  LeeEllen  Friedland:  Front-End  Validation  of  Data  for  Micro-Social  HSCB  Models 

•  Current  work  is  pending  approval  of  the  new  SOW 


January  2011 

•  Created  Development  Scenario  for  project  year  two,  CDRL  A001  (update) 

o  Submitted  with  this  report 

•  Updated  the  project  Statement  of  Work  to  reflect  expected  resources  for  March-October  2011 

o  Submitted  separately  by  email  to  Dr.  Ivy  Estabrooke,  cc:  Gary  Kollmorgen  and  Clark 
Phillips 

o  Includes  updated  spend  plan,  also  submitted  with  the  SOW  by  email 
December  2010 

•  Caught  up  invoicing  against  funding  increment  received  on  November  30,  2010 

•  Started  work  on  updated  spend  plan  and  deliverables  schedule  to  accommodate  2010  funding 
gap 

November  2010 

•  Project  activity  has  been  on  hold  pending  funding  for  project  year  two 

•  On  November  30,  received  new  funding  increment  covering  at-risk  work  executed  in 
September  and  October  2010 

October  2010 

•  Prepared  submission  for  exhibition  booth  at  HSCB  Focus  2011  conference 

•  Other  project  activity  is  on  hold  pending  funding  for  project  year  two 

•  Please  note  the  updated  contact  information  for  our  financial  team:  Izett  Barnett,  Director  of 
Finance  (ibamett@alelo.com) 

September  2010 
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•  Participated  in  ONR  Capabilities  Open  House,  September  20-21,  2010 

•  Other  project  activity  is  on  hold  pending  funding  for  project  year  two 

August  2010 

•  Completed  year  1  Technical  Performance  Evaluation  (TPE)  in  Arlington,  VA 

•  Started  preparations  for  ONR  Capabilities  Open  House  in  September  2010 


July  2010 

•  Completed  year  1  evaluation  and  submitted  evaluation  report  (CDRL  A005) 

•  Submitted  update  to  TCA  design  doc  (CDRL  A003) 

•  Submitted  update  to  culture  model  document  (CDRL  A002) 

June  2010 

•  Implemented  test  cases  that  can  be  run  in  the  framework  completed  last  month 

•  Continuing  development  of  behavior  rules  to  pass  these  test  cases 

•  Continuing  development  work  for  Culture  Model  and  TCA  Design 

•  Outlined  year  1  evaluation  report  and  assigned  writing  tasks 

May  2010 

•  Updated  Model  Design  Document  and  Evaluation  Design  Document  from  April  2010 
deliverables,  according  to  follow-up  discussion  with  Dr.  Ivy  Estabrooke  (attached) 

•  Introduced  and  trained  new  student  intern  who  will  work  on  the  evaluation  team  between 
May  and  August,  2010 

•  Implemented  testing  framework  to  support  evaluations  outlined  in  CDRL  A004 
April  2010 

•  Prepared  and  submitted  updates  to  the  Microsocial  Model  Document  (CDRL  A002)  and 
TCA  Design  Document  (CDRL  A003)  on  schedule,  on  April  30,  2010 

•  Prepared  and  submitted  Year  1  Evaluation  Criteria  (CDRL  A004),  on  schedule,  on  April  30, 
2010 

•  Annotated  test  dialogs  with  gold-standard  logical  axioms  for  year  1  evaluation 

•  Updated  the  publication  inventory  (submitted  with  this  report) 

March  2010 

•  Continued  data  development  on  interview  material  from  SME2 

•  Identified  test  dialogs  for  year  1  evaluations  and  defined  test  cases  over  these  dialogs 

•  Started  implementation  of  the  test  cases  and  preparation  of  the  year  1  evaluation  criteria 
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document 

•  Continued  implementation  work  for  TCAs  and  preparation  for  the  updated  TCA  document 
due  in  April 

•  Continued  extension  of  the  microsocial  model  and  preparation  for  the  updated  model  design 
document  due  in  April 

•  Inventoried  publications  that  have  been  produced  on  this  project  so  far  (submitted  with  this 
report) 

February  2010 

•  Prepared  for  and  presented  6-month  review  with  ONR  in  San  Francisco,  CA  on  February  9th 

•  Prepared  a  revised  budget  with  justification  and  submitted  to  ONR  on  February  18th 

•  Prepared  an  updated  revised  budget  with  new  FY2010  spend  target  of  $250,000;  budget  is 
now  under  review  by  financial  team  and  will  be  submitted  this  week  (before  3/12/2010) 

•  Data  collection:  performed  data  development  on  interview  material  from  SME  2 

January  2010 

•  Data  collection:  conducted  initial  interview  with  SME  2,  screened  candidates  for  SME  3 

•  Started  implementation  work  based  on  the  TCA  design  document  delivered  in  December 

•  Prepared  materials  for  6-month  review  meeting  in  San  Francisco,  CA  in  early  February  2010 

December  2009 

•  Delivered  first  draft  of  culture  model  document  (CDRL  A002)  on  December  24,  2009 

•  Delivered  first  draft  of  TCA  design  document  (CDRL  A003)  on  December  24,  2009 

•  Started  scheduling  for  6-month  review  meeting  in  San  Francisco,  CA  in  early  February  2010 

November  2009 

•  Reviewed  Data  Collection  process  with  Alelo  Culture  Team  and  Dr.  Agar;  revised  Data 
Requirements  Document  (DRD);  extracted  sample  dialogs  from  dry-run  data  collection 
executed  in  October;  created  data-collection  schedule  for  December  2009-March  2010: 

o  December  2009:  Polish  Interview  Protocol  Document  (IPD)  with  input  from  Mike 
Agar;  conduct  follow-up  interview  with  SME  1  (dry-run  SME);  complete  the 
technical  design  document  and  culture  model;  schedule  phone  conference  with  ONR 
on  cross-cultural  competencies. 

o  January  2010:  Extract  dialogs  from  follow-up  interview  with  SME  1;  choose  2 
additional  SMEs  and  schedule  initial  and  follow-up  interviews, 
o  February-March  2010:  conduct  interviews  and  extract  20-30  dialogs  from  SME  2, 
SME  3;  identify  development  and  test  dialogs  for  year  1  evaluations;  update  dialogue 
system  and  design  documentation  based  on  this  data. 

•  Created  first  draft  of  culture  model  document  for  deliverable  in  December,  2009 

•  Implemented  prototype  of  text -based  conversational  agent  (TCA)  dialog  loop  and  created 
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first  draft  of  design  document  for  deliverable  in  December,  2009 

•  Prepared  article  for  Scientist  spotlight  in  January  HSCB  Newsletter 

October  2009 

•  Held  quarterly  in-person  meeting  with  all  subcontractors  and  team  members  on  October,  27 
2009  in  Los  Angeles 

•  Completed  in-house  review/collection  of  cultural  data  resources  for  Afghanistan  with  their 
IRB -relevant  information 

•  Created  a  Data  Requirements  Document  (DRD)  to  describe  the  needs  for  additional  data 
collection 

•  Created  a  draft  Interview  Protocol  Document  (IPD)  that  defines  procedures  for  collecting 
data  described  in  the  DRD  in  interviews  with  Subject-matter  experts 

•  Executed  a  dry-run  of  the  IPD  with  Alelo  consultants  serving  as  native- speaker  language 
experts  in  Pashto 

•  Continued  work  on  alignment  between  RACCA  WG  cultural  learning  objectives  and  project 
objectives 

September  2009 

•  Held  kick-off  meeting  at  ONR  on  8  September,  2009 

•  Started  in-house  review/collection  of  cultural  data  resources  for  Afghanistan  with  their  IRB- 
relevant  information 

•  Initiated  negotiation  of  an  appointment  to  observe  a  lane  training  session  at  29  Palms  in  order 
to  collect  additional  first-person  accounts  and  sample  dialogs  from  human  role -players 

•  Received  documentation  of  the  RACCA  WG  cultural  learning  objectives  from  Dr.  Allison 
Abbe  and  started  work  on  alignment  between  these  objectives  and  the  objectives  for  the 
CultureCom  project 

•  Continued  to  perform  background  cultural  research  and  work  on  the  first  draft  of  axioms  and 
concepts  for  the  culture  model  that  supports  our  Year  1  Development  Scenario 

August  2009 

•  Completed  contract  negotiations  with  project  start  date  31  July,  2009 

•  Selected  geographic  area  of  focus  and  tasks  of  focus  for  year  1 

•  Created  and  submitted  Year  1  Development  Scenario  (CDRL  A001) 

•  Prepared  materials  for  kick-off  meeting  08  September,  2009 

2  OUTSTANDING/ANTICIPATED  ISSUES  AND  SOLUTIONS  TO 
THOSE  ISSUES 

February  2011 

•  Current  funding  ends  in  March 
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•  Continued  work  on  the  project  is  pending  approval  of  the  updated  SOW 

o  Along  with  new  funding  increment  to  support  the  work  described  in  SOW 

January  2011 

•  Continued  work  on  the  project  is  pending  approval  of  the  updated  SOW  and  receipt  of  a  new 
funding  increment  to  support  the  work  described  there 

December  2010 

•  The  project  schedule  is  likely  to  be  delayed  by  90-120  days  total  in  the  revised  project  plan, 
which  accounts  for  work  stoppage  due  to  funding  gaps 

November  2010 

•  The  project  schedule  is  likely  to  be  delayed  by  90-120  days  total  in  the  revised  project  plan, 
which  accounts  for  work  stoppage  due  to  funding  gaps 

October  2010 

•  The  project  schedule  is  likely  to  be  delayed  an  additional  30  days  (60-90  days  total)  once 
activity  resumes,  pending  year  two  funding 

September  2010 

•  The  project  schedule  is  likely  to  be  delayed  an  additional  30  days  (60-90  days  total)  once 
activity  resumes,  pending  year  two  funding 

August  2010 

•  The  schedule  for  project  year  two  will  be  delayed  by  30-60  days,  due  to  the  following  issues 

o  Insufficient  time  was  planned  in  the  August  schedule  to  accommodate  year-end  travel 
and  reporting  for  year  one  along  with  year-two  kickoff  activities;  however,  the  year- 
end  activities  were  worth  the  investment  in  time  since  they  resulted  in  helpful 
communication  with  the  sponsors  that  we  will  apply  going  forward 
o  The  current  funding  increment  has  ended  (at  time  of  writing  we  have  approximately 
two  weeks  left,  depending  on  level  of  effort) 
o  We  are  moving  our  staff  to  other  projects  for  the  time  being,  but  we  plan  to  be  ready 
to  start  CultureCom  work  again  as  soon  as  funding  permits 


July  2010 

•  No  known  issues 
June  2010 


Copyright  ©  2011  Alelo  TLT,  LLC 

Use  or  further  disclose  of  the  information  in  this  document  is  prohibited  unless  specifically  authorized  5 


Alelo  TLT,  LLC 
Project:  HSCB  CultureCom 


Contract:  N00014-09-C-0613 
Monthly  Report  for  February  2011 


•  No  known  issues 
May  2010 

•  No  known  issues 
April  2010 

•  No  known  issues 
March  2010 

•  No  known  issues 
February  2010 

•  No  known  issues 
January  2010 

•  No  known  issues 
December  2009 

•  No  known  issues 
November  2009 

•  We  have  not  yet  received  invoices  from  the  University  of  Southern  California,  a 
subcontractor  on  this  project.  As  a  result  the  hours  worked  by  USC  have  not  yet  been 
charged  to  the  spend  plan.  However  we  do  expect  to  receive  an  invoice  in  the  month  of 
December  which  will  correct  our  spending  with  respect  to  the  plan. 

October  2009 

•  Alelo  resolved  the  spend  plan  issue  from  last  month  by  completing  a  revised  spend  plan; 
submitted  to  ONR  and  SA,  Inc.  by  email  on  26  October,  2009 

•  We  have  not  yet  scheduled  a  phone  conference  with  ONR  on  the  topic  of  cultural  learning 
objectives;  We  engaged  or  collaborators  on  this  project,  both  subcontractors  (Dr.  Michael 
Agar)  and  other  colleagues  (Dr.  LeeEllen  Friedland)  in  discussions  of  this  topic  and  we  are 
ready  to  schedule  the  conference  for  the  month  of  November  at  ONR’s  convenience 

September  2009 

•  Spend  plan  should  be  revised  and  we  have  already  started  this  work;  original  spend  plan 
assumed  a  constant  rate  of  expenditures,  when  in  fact  the  costs  during  the  initial  months  will 
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be  lower  while  costs  during  software  development  and  integration  months  will  be  higher;  we 
are  working  on  a  revised  spend  plan  that  reflects  this  change  in  our  method  of  estimation  as 
well  as  new  hires  that  we  plan  to  take  on  for  the  upcoming  software  development  phase 

August  2009 

•  No  known  issues 

3  FUTURE  SIGNIFICANT  EVENTS 

February  2011 

•  Pending  approval  of  the  new  SOW: 

o  Recruitment  of  Subject  Matter  Experts  for  ethnographic  interviews 
o  Technical  review  of  results  from  year  one  and  start  of  design  work  for  year  two 
prototype 

o  Background  research  and  information  gathering  by  the  culture  modeling  team 
January  2011 

•  Internal  kick-off  scheduled  for  February  3-4  in  Los  Angeles,  with  all  team  members 

•  Project-related  presentations  will  be  given  at  HSCB  Focus  2011  Conference  February  7-10 

•  Future  work  is  pending  approval  of  the  new  SOW 

o  Recruitment  of  Subject  Matter  Experts  for  ethnographic  interviews 
o  Technical  review  of  results  from  year  one  and  start  of  design  work  for  year  two 
prototype 

o  Background  research  and  information  gathering  by  the  culture  modeling  team 
December  2010 

•  Project-related  presentations  will  be  given  at  HSCB  Focus  2011  Conference  in  February 

•  Updated  project  plan  is  still  under  development  to  identify  year  two  activities  that  may  be 
funded  with  current  increment 

•  Project  year  two  internal  kick-off  is  likely  to  occur  in  January  201 1 

November  2010 

•  Updated  project  plan  will  be  produced  in  December  to  identify  year  two  activities  that  may 
be  funded  with  current  increment 

•  Project  year  two  internal  kick-off  is  likely  to  occur  in  January  201 1 
October  2010 

•  Project  year  two  internal  kick-off  may  occur  in  November  or  December  2010 
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September  2010 

•  Project  year  two  internal  kick-off  may  occur  in  November  2010 
August  2010 

•  Dry-run  on  September  10th  for  the  ONR  Capabilities  Open  House 

•  Pending  dry-run  results,  ONR  COH  on  September  20th-21st 

July  2010 

•  Technical  Performance  Evaluation  (TPE)  will  be  held  at  MITRE  on  August  1 1,  2010 
June  2010 

•  Remaining  milestones  (internal  and  external)  for  July: 

o  Populate  a  series  of  test  cases  from  the  Year  1  evaluation  dialogs  (Completed) 
o  Populate  TCA  rule  instances  based  on  these  test  cases  (7/8) 

o  Calculate  metrics  from  the  evaluation  criteria  document  over  these  test  cases  (7/16) 
o  Generate  and  submit  report  documents  with  evaluation  results  (7/31) 

May  2010 

•  Next  external  milestone  is  evaluation  results  with  a  year-1  final  report  in  July,  2010 

•  Internal  milestones  for  June  and  July: 

o  Populate  a  series  of  test  cases  from  the  Year  1  evaluation  dialogs  (6/24) 
o  Populate  TCA  rule  instances  based  on  these  test  cases  (7/8) 

o  Calculate  metrics  from  the  evaluation  criteria  document  over  these  test  cases  (7/16) 
o  Generate  and  submit  report  documents  with  evaluation  results  (7/31) 

April  2010 

•  A  student  intern  will  join  our  team  starting  in  May,  2010.  His  work  will  focus  on 
implementation  of  the  evaluation  dialogs  using  TCA  behavior  rules,  and  on  supporting  our 
year  1  evaluation  experiments  and  written  reports. 

March  2010 

•  Next  milestone  is  evaluation  criteria  with  updated  TCA  design  and  culture  model,  due  April 
30,  2010 

February  2010 

•  Next  milestone  is  evaluation  criteria  with  updated  TCA  design  and  culture  model,  due  April 
30,  2010 
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January  2010 

•  6-month  review  meeting  scheduled  for  Tuesday,  February  8th  in  San  Francisco,  CA. 

December  2009 

•  We  have  been  in  contact  with  Dr.  Allison  Abbe  regarding  our  follow-up  phone  conference  on 
cross-cultural  learning  objectives  and  we  hope  to  have  an  opportunity  to  talk  with  her  this 
month  (January  2010). 

November  2009 

•  Drafts  of  the  Culture  Model  and  the  TCA  Design  document  are  due  to  be  submitted  to  ONR 
in  December,  2009 

•  Please  note:  the  Alelo  address  and  phone  number  have  changed.  Updated  contact 
information  is  given  on  the  cover  page  of  this  Monthly  Report,  and  repeated  here  for 
convenience: 

Alelo 

12910  Culver  Blvd.  Suite  J 

Los  Angeles,  CA,  90066 

Telephone  (for  Alicia  Sagae):  310-574-7528 

October  2009 

•  Current  data  collection  efforts  using  the  DRD  and  IPD  described  in  Section  1  should  result  in 
additional  example  dialogs  becoming  available 

•  These  dialogs  will  be  used  to  focus  our  current  development  work  on  the  Culture  Model  and 
the  Text-based  Communicative  Agents  Design  Document  (TCA  Design);  drafts  of  these 
documents  are  due  to  be  submitted  to  ONR  in  December,  2009 

September  2009 

•  A  visit  to  the  live  training  simulations  at  29  Palms  will  hopefully  take  place  in  October  or 
November  2009. 

•  An  internal  project  meeting  with  all  collaborators  is  scheduled  for  October  26,  2009. 

August  2009 

•  The  project  kick-off  meeting  at  Strategic  Analysis  scheduled  for  08  September,  2009. 

4  TOTAL  MONTHLY  BILLED  EXPENDITURE  DOLLAR 
AMOUNTS  AND  INCLUDE  COPIES  OF  “ACTUAL  INVOICES” 
SENT  FOR  PAYMENT  WITH  THIS  REPORT. 
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Alelo  TLT,  LLC 
Project:  HSCB  CultureCom 


Contract:  N00014-09-C-0613 
Monthly  Report  for  February  2011 


Expenditures  for  the  month  of  February  201 1 
FY  2010:  $25,000  (estimate) 


Expenditures  for  the  month  of  January  201 1 
FY  2010:  $6,565 


Expenditures  for  the  month  of  December  2010 
FY  2010:  $1,356 

Expenditures  for  the  month  of  November  2010 

Work  at  risk  (self-funded,  will  invoice  if  additional  funding  becomes  available):  $21 1 
Expenditures  for  the  month  of  October  2010 

Work  at  risk  (self-funded,  will  invoice  if  additional  funding  becomes  available):  $636 
Expenditures  for  the  month  of  September  2010 

Work  at  risk  (self-funded,  will  invoice  if  additional  funding  becomes  available):  $12,667 

Expenditures  for  the  month  of  August  2010 
FY2010  (ACRN  AB):  $38,765 
FY2009  (ACRN  AA):  $70,449 


Expenditures  for  the  month  of  July  2010 
FY2010:  $32,935 

*This  invoice  does  not  include  expenses  for  June  or  July  for  our  subcontractor,  the  University  of  Southern 
California.  We  continue  to  pursue  timely  invoicing  from  USC.  Our  current  estimate  of  the  un-invoiced 
cost  is  $17,024. 


Expenditures  for  the  month  of  June  2010 
FY2010:  $31,851 
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Alelo  TLT,  LLC 
Project:  HSCB  CultureCom 


Contract:  N00014-09-C-0613 
Monthly  Report  for  February  2011 


*This  invoice  does  not  include  expenses  for  our  subcontractor,  the  University  of  Southern  California. 
July  expenditures  for  USC  will  be  combined  with  August,  2010  expenditures  due  to  a  delay  in  USC  posting 
invoices  to  Alelo. 

Expenditures  for  the  month  of  May  2010 
FY2010:  $21,340 

Expenditures  for  the  month  of  April  2010 
FY2010:  $48,011 

Expenditures  for  the  month  of  March  2010 
FY2010:  $67,143 

Expenditures  for  the  month  of  February  2010 
FY09:  $71,588.99 

Expenditures  for  the  month  of  January  2010  (Note:  total  billed  amount  is  under  budget  at  this  time  due 
to  delay  in  receiving  invoices  from  all  subcontractors) 

FY09:  $79,759.90  (January  and  December) 

Expenditures  for  the  month  of  December  2009 

FY09:  $0 

Expenditures  for  the  month  of  November  2009  (Note:  total  billed  amount  is  under  budget  at  this  time 
due  to  delay  in  receiving  invoices  from  all  subcontractors) 

FY09:  $15,791 

Expenditures  for  the  month  of  October  2009 
FY09:  $9,531 

Expenditures  for  the  month  of  September  2009 
FY09:  $12,020 

Expenditures  for  the  month  of  August  2009 
FY09:  $4,446 
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